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In quasi low dimensional superconductors under parallel magnetic fields applied along a conduct- 
ing direction, vortex lattices with a modulation of Fulde-Ferrell-Larkin-Ovchinnikov (FFLO) type 
perpendicular to the field may occur due to an enhanced paramagnetic depairing. As the strength 
of an in-plane field is varied in a Q2D material, the Josephson vortex lattices accompanied by nodal 
' planes are formed in higher Landau level (LL) modes of the superconducting (SC) order parameter 

and show field-induced structural transitions. A change of orientation of nodal planes induced by 
these transitions should be observed in transport measurements for an out-of-plane current in real 
superconductors with point disorder effective on the SC layers. Further, the // C 2-transition from this 
higher LL state to the normal phase is of second order for moderately strong paramagnetic effects 
but, in the case with a strong enough paramagnetic effect, becomes discontinuous as well as the 
transition between this modulated state and an ordinary Abrikosov vortex lattice in the lowest LL. 
Relevance of these results to recent observations in organic superconductors suggesting the presence 
of an FFLO state are discussed. 
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Recent experimental evidences of a new type of high field phase in a heavy fermion superconductor CeCoIns have 
led to a new occasion on studies of the so-called Fulde-Ferrell-Larkin-Ovchinnikov (FFLO) superconducting (SC) state 
C/3 \ In CeCoIns in H | ab, the features of phase diagram and a change of elastic response through the Abrikosov 

to FFLO transition were consistent with the picture based on the presence of an FFLO modulation parallel to the 
applied field H (and nodal planes perpendicular to H) 0, ||. Reflecting the fact that the Fermi surface relevant 
to the superconductivity is not purely cylindrical in CeCoIns, the corresponding state has also been observed in H 
perpendicular to the layers (H || c). [4] This FFLO vortex state modulating along H has no additional variation of 
the vortex structure in the plane perpendicular to H and is, just like the ordinary Abrikosov lattice, well described 
in the lowest Landau level (n = LL) where no spatial variation other than the field-induced vortices occurs as far 
q | as the SC order parameter is a single scalar field. [5|, |6j Actually, any vortex state stable in the n = LL is isotropic 
l—— '■ in character in the plane perpendicular to H. 

Inevitably, possible FFLO states modulating in a direction perpendicular to the field have to be described in terms 
of higher LL modes of the SC order parameter. When imagining an artifitial situation in which the field strength 
' inducing the formation of vortices is much weaker than the total magnetic field associated with the Zeeman energy, 
CO , it is a higher LL state which determines the i? C 2(T)-line and describes the vortex state just below it 0]. It has been 
recently noticed that even the 3D high field vortex lattice may become such a higher LL state in clean limit and at 
low enough temperatures [f|. Although such a state is easily lost if a finite quasiparticle damping is not negligible 
it may be realized in superconductors with strong enough paramagnetic depairing. Among them, the n = 1 LL 
state closer to the n = LL is the most relevant to real systems. It has a one-dimensional stripe pattern of nodal 
planes appearing separately from the vortices @, Q and is expected to have a peculiar property that directional probes 
such as the transport measurements become anisotropic depending on the orientations of the nodal planes relative to 
J> , the crystal lattice. When considering the striped vortex lattices in layered systems in fields parallel to the layers, a 
question arises: A typical Fermi surface of such materials, i.e., the cylindrical one, suggests that the FFLO modulation 
tends to become parallel to the layers, accompanied by nodal planes vertical to the layers, while the nodal planes tend 
to be pinned, as well as the vortices, by the layer structure so that the layered structure favors nodal planes oriented 
along the layers. In considering a possible FFLO-like state in quasi 2D materials, it is necessary to correctly resolve 
such a competition in the order parameter structure. 

In this work, we have studied the n — 1 LL Josephson vortex lattices occurring in the layered system in parallel 
fields in the situations where, due to a strong paramagnetic depairing, this state is realized below H C 2(T). We find 
that, as the field is varied, the orientation of nodal planes changes accompanying structural transitions of the vortex 
lattice itself. Since, in real materials, the pinning effect due to the point disorder on the SC layers is effective especially 
for nodal planes not parallel but perpendicular to the layers, such changes of orientation of nodal planes should affect 
the resistivity for currents perpendicular to the layers. Relevance of these results to the observation Q in the quasi 
2D organic field-induced superconductor A-(BETS)2FeCi4 will be discussed. In addition, possible phase diagrams in 
cases with the n = 1 LL state in the parallel fields will also be discussed, and we point out that the ./^-transition 
(i.e., the mean field SC transition) is of second order for reasonable values of the Maki parameter, while it becomes 
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of first order for very high but, nevertheless, realistic values of the Maki parameter. This result may be relevant to 
the recent report of heat capacity data of a k-(ET) 2 organic material 

We start from the same BCS model as in Ref.[6j for quasi 2D systems which includes the Zeeman energy and the 
interlayer hopping energy terms 
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where j is the index numbering the SC layers, d is the interlayer spacing, and ap, B H = p B H or —/j,bH is the Zeeman 
energy. In discussing our calculation results, the strength of the paramagnetic effect will be measured by the Maki 
parameter, i.e., the ratio between the orbital and Pauli limiting fields, which will be defined here by the quantity 
olm = Mb-H^d v^)/^bT c . The conventional Maki parameter in H || c is obtained by multiplying a constant factor 
~ 7.0 to this ctM, where H^ b \o) ~ l/(2e£g) is the orbital limiting field in 2D limit, and £o is the in-plane coherence 
length. Hereafter, the applied field H is directed to the x-axis parallel to the SC layer. 

In studying nearly 3D-like superconductors in which the out-of-plane coherence length £ c (0), which will be defined 
later, is longer than d/y/2, the interlayer hopping energy term cx J is treated on the same footing as the in-plane 
kinetic energy term, and, instead, effects of the discrete layered structure on the SC order parameter are not well 
incorporated in the GL description [1, d, @|. Since this layering effect on the SC order parameter is one of the 
main concerns in this work, we choose here rather to treat J perturbatively. When the SC order parameter belongs 
primarily to the n-th LL, the resulting GL free energy in the mean field approximation takes the form 



J%> = d^Jdy 
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similar to the familiar Lawrence-Doniach model, where An implies the projection of A^ into the n-th LL, and 
n s = —\d/dv s + 2eA(r s ). If the iJ c2 -transition is discontinuous, 0(|A^| 6 ) term omitted in eq.© needs to be 
incorporated. Microscopic details in the vortex states are reflected altogether in the coefficients such as a n and c n . 
In eq.([2]), a possible modulation parallel to H in A„ was neglected. Possibilities of this modulation need to be 
incorporated in considering phase diagrams and will be discussed at the end of this paper. The LL representation of 
the order parameter can be used for the layered system by rewriting [ll|, 03 
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Jjjz in the form 
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That is, spatial variations of A 1 ^} on the SC layers are described in terms of the continuous order parameter A n (y, z). 
Hereafter, the linear gauge A = —Hzy will be used. 

The coefficients will be treated in the same manner as in Ref.[(|. The coefficients a n and c„ of the quadratic term 
are given by 
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and A p denotes the normalized orbital part of the pairing function satisfying (|A p | 



1. In the expression of p, the 



small imaginary part w z (= 0{J / 'Ep)w y ) was introduced as a cutoff to avoid a possible failure of the perturbation in 
J, although we find that, except in the close vicinity of T = 0, teh presence of such a small imaginary term does not 
lead to any quantitatively visible contribution. Hereafter, fi will be replaced by w y /(^/2T c rH)- 

Next, A n (y,z) will be expressed in terms of the Abrikosov lattice solution generalized to the n-th LL and 

commensurate to the layer structure, in the form aW$|* , where 
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it — k/(2eH) = wd. Then, Jjjl can be rewritten in the form 
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and p = n d /r 2 H — 2n 2 ed H. The positive constant @A,n wm be defined later. 

If the i? c2 -transition is of second order, the i? c2 (T)-curve consists of a sequence of those satisfying A 2> „ = at the 
highest field. However, one should note that, if Vi <n < on such a line determined by A 2 .„ = 0, the 7J c2 -transition is 
discontinuous and should lie just above the line A 2i „ = 0. The coefficient Vi :n (n — or 1) is given by 



Vi,n = 3^ dp 1 dp2dp 3 f(^^p^(^\A p \ i p n (p J )exp(^~^fi 2 (^2p J 



where po = 1 , and 
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In writing down eqs.([9]) and (| 1 2[) . wavenumber dependences leading to a spatially nonlocal interaction between the 
SC order parameters were neglected Q. Such a nonlocality might have brought a subtle change of vortex lattice 
structure. Hence, this simplification corresponds to assuming that such an effect of nonlocality is much weaker than 
the effect of the layering on the vortex lattice structure. 

Let us first examine (^/-dependences of the mean field i? c2 -line. Typical i? c2 -curves are shown in Figs.l and 2 
(a) by assuming the i? c2 -transition to be of second order. As Fig.l shows, when olm is small enough, the -ff c2 -curve 
increases with a positive curvature upon cooling, reflecting the confinement of vortices occurring between the interlayer 
spacings in higher fields when £ c o < dj\/2 [ll|,[l3j]. The characteristic field H cr beyond which this confinement begins 
to occur is given by 
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where the anisotropy 7 = £o/£c(0) is conventionally defined in the usual GL region near T c in terms of eq.([7]) in low 
field limit by 
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FIG. 1: Second order //^(T^-curves (thin solid curves) denned in the n = LL for aj; = 0.001, 0.01, and 0.03. Here, 
h — H/H^^ (0), and t — T/T c (0). The dotted line is the extrapolation of the thin solid curve for ccm — 0.03, and the role of 
the mean field transition //^-transition line is played by a first order transition curve starting from t = 0.275 indicated by the 
solid arrow and lying at higher fields. The thick solid curve is the fictitious instability curve in n = 1 LL for o.m = 0.03. 
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FIG. 2: (a) Real and virtual second order // C 2(T)-transition curves for o>m — 1.0 and 1.5. For au = 1-0, the real //^-transition 
is of second order in t < 0.3, and the thin dotted curve is the virtual second order transition curve in n — LL and is preceded 
by a first order //^-transition curve lying in slightly higher fields in 0.3 < t < 0.5. The flat thick solid curve in <xm = 1.5 is 
also preceded by a first order Z/ C 2-transition curve in n = 1 LL elevating upon cooling, (b) The corresponding 1/4,1 v.s. t curves 
in ctM = 1.0 and 1.5. 

In the figures, h cr = H cr /H!^ h \o) ~ 3.0. If H C 2(0) > H CI due to a large Hp and/or a large 7, the saturation of H C 2 
due to the paramagnetic effect at lower temperatures coexists, as in the ctM = 0.01 case in Fig.l, with the layering- 
induced positive curvature of H C 2 (T)-line at higher temperatures. In this case, the limiting of superconductivity 
occurs in the range where the vortices are inactive because they are confined within the interlayer spacings. Then, 
the approximation in the Pauli limit neglecting the presence of vortices at low temperatures may be useful. However, 
when H C 2(0) > H cr , the discontinuous H C 2 transition and the FFLO state in n — LL do not easily occur even at low 
enough temperatures. Actually, the //^-transition in the ajf = 0.01 curve remains continuous even in low T limit, 
and the n = 1 LL state is never realized there. In the intermediate case, ajj = 0.03, with ZZ c2 (0) comparable with 
H cr , the //^-transition becomes discontinuous in t < 0.29. Still, the n = 1 LL instability line (the thick solid curve in 
Fig.l) lies at lower fields so that the n = 1 LL state with modulation perpendicular to H does not occur. In contrast, 
if H cr > ZZ C 2(0) due to a large ajj and/or a smaller 7, the limiting of superconductivity occurs in the field range where 
a slight change of the magnetic field results in structural transitions between different vortex lattices [HI, [HJ ■ In other 
words, the presence of vortices cannot be neglected in H CT > H C 2 (0) even within the mean field approximation. As in 
Fig. 2 (a), the H C 2 curve in this case does not show a portion with a positive curvature at intermediate temperatures. 

In contrast, in Fig. 2 where H cr > ZZ C 2(0), the n = 1 LL vortex state becomes dominant at lower temperatures : As 
Fig. 2 (a) shows, the n = 1 LL modes determine H C 2(T) and the vortex state just below it in t < 0.3 (t < 0.4) for 
oim = 1.0 (cym = 1-5). Further, the corresponding P4.1 curves shown in Fig. 2 (b) imply that, for cxm = 1-0, the mean 
field H C 2 transition is of second order in most of temperatures, while it is rather a discontinuous one for olm = 1-5. 
Just like the H C 2 line in n = LL, the //^-transition between the n = 1 LL state and the normal phase tends to 
become discontinuous with increasing a m ■ Since the structural transition between the n = 1 LL state and any vortex 
state in n — LL is inevitably of first order due to the absence of a continuity between their structures, there are two 
first order or discontinuous transitions in the high field range for the olm = 1.5 case. 

Now, let us turn to examining possible vortex lattice structures in the n = 1 LL state by focusing on the situations, 
including the au = 1-0 case in Fig. 2, with a second order H C 2 transition. Then, the structure with the lowest value 
of the positive quartic term of T has the lowest energy when a vortex lattice is described in a single LL. Under 
the assumption neglecting spatial nonlocalities in the quartic term mentioned above, a stable lattice structure of 
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FIG. 3: Schematic figure expressing the Josephson vortex lattices changing with sweeping the magnetic field p. The vortex 
lattices in the dark regions are accompanied by nodal lines perpendicular to the SC layers (see the figure A in Fig.4), while the 
nodal planes are parallel to the layers, as in the figure C of Fig.4, in the remaining p-ranges. 

Josephson vortices at each p is determined by the generalized Abrikosov factor 

Em ex P f Vnmz/d ) | Ai (y, z) 
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By substituting Ai = ai^^ into eq. tfll))) . it becomes 
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where Bq = 1, 

Bx(m, sx,s 2 ) = -(3 - 671-V 1 ™ 2 - 2w 2 p(s 2 1 + si) + (w 2 p( Sl - s 2 ) 2 + TT 2 p- 1 m 2 )(w 2 p(s 1 + s 2 ) 2 + 7r 2 p _1 m 2 )) (18) 



A sequence of stable lattice structures following from this /3a, i is shown in a fixed window of p values in Fig. 3. In 
contrast to the Josephson vortex lattices constructed in n = LL [ll|, [ll| where diferent lattices are distinguished by 
the w-values, differences in the orientation of nodal planes provide an additional characterization of differerent n = 1 
LL states. The orientation of nodal planes are directly visible in the amplitude of the SC order parameter in n = 1 
LL which is given by 

lAx^z)! 2 = ^E m+s (l-hm 2 (w 2 p) + 7r 2 s 2 (pw 2 )- 1 ))cos(^m{m + s) 
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x expl — - I m 2 (w 2 p) H T )) cos(2n( my + sz)), (19) 

V 4 V pw 2 ) J 

where y = wdy / (2TTrf I ), and z — z/(2wd), and the sign factor E n is 1.0 (0) for an even (odd) n. As shown in Fig. 4(a), 

the i&j^ solution at a fixed w (> 1) value can become a structure with the lowest energy at two p values. At the 
low p value, the nodal planes are perpendicular to the layers, while they are oriented along the layers at the higher 
p value. In general, the nodal planes tend to orient along the direction with a shorter inter-vortex spacing. Since, in 
the layered system with two-fold anisotropy, the layer structure favors the orientation of vortices parallel to the layers 
in higher fields, the orientation of nodal planes is affected by the strength of the applied magnetic field. Note that, 
as Fig. 4(b) shows, the transformation between these two structures at a fixed w does occur not through a rotation of 
nodal planes but via some merger between the vortices and the nodal planes. Such an intermediated state, Fig. 4(b), 
composed only of the nodal planes is not realized for w > 1 due to a structural transition to a state with a different 
w-value. However, in the w = 1 state where all of the interlayer spacings are occupied by the vortices, it is realized : 
In this case, the nodal planes do not become parallel to the layers because such nodal planes parallel to the SC layers 
in w — 1 case would imply a vanishing of a± on the SC layers and would lead to a strong energy cost. Consequently, 
the ground state structure in p > 0.7 where only the w = 1 state is realized is that of Fig. 4(b). It is interesting to 
point out that this structure is a kind of square lattice composed only of the nodal planes and similar to the ground 
state expected in the model in the vortex free Pauli limit [14[ . 
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FIG. 4: The p-dependence of /3a, i of the w — 6 Josephson vortex lattice (see eq.(|8}) determining the relative stability of the 
lattices. The figures (A), (B), and (C) express |Ai(j/,z)| 2 at three points in the first figure, respectively. 

As shown elsewhere, a misfit from a commensurability condition leads to an energy gain by rotating a symmetry 
axis of vortices from the layers' orientation. In contrast to the n — LL case with no nodal planes, however, such 
a rotated solid does not lead to lowering of energy in the presence of the additional nodal planes. This is due 
partially to the fact that, in higher fields, a pinning of nodal planes due to the layer structure leads to an energy 
gain. Since this pinning due to the layering overcomes the tendency of rotation induced by a misfit, the energy of 
each rotated state is almost degenerate with the corresponding nonrotated one. This is why we have focused on 
the nonrotated structures in the figures, although their inclusion does not change our interpretation of experimental 
observations mentioned below. 

Finally, let us discuss relevance of results given above to recent observations in organic superconductors with strong 
anisotropy suggestive of the presence of an FFLO state 0, As is clear from Fig. 2, the expected phase diagram in 
the present situation with strong anisotropy is not universal. The assumed strong anisotropy suggests that the vortex 
tilt modulus in the FFLO state in n = LL with a modulation parallel to H will be significantly reduced so that 
this state may be fragile . 6| . Then, the discontinuous nature of of the iJ C 2-transition between this FFLO state and 
the normal phase may be changed into a continuous crossover (2j. The transition between the Abrikosov and FFLO 
states in n = LL, appearing in weakly anisotropic cases, is of second order as far as the quasiparticle's lifetime 
is long enough 0, ||. However, this FFLO state realized in CeCoIns may be preceded by the n = 1 LL state and 
thus, may not occur in the case with strong anisotropy. In contrast to this, the transition between the n = 1 LL 
state and the n = LL states is, as mentioned earlier, of first order. As indicated through Fig. 2, the character of 
the i?e2-transition to the n = 1 LL modulated state depends on the magnitude of ajvf and possibly, also on 7. In 
particular, the resulting coexistence of the two discontinuous transitions in the case with large enough ajf seems to 
be consistent with the recent observation of two transitions accompanied by a hysterisis [151 ] and a sharp peak of heat 
capacity in k-(ET)2 Cu(NCS)2 [lOj. If so, the high field phase at lower temperatures should have a one-dimensional 
modulation perpendicular to the field. On the other hand, it is possible [J| that the transition between the Abrikosov 
lattice and an FFLO state in n = LL modulating along H is of first order in the case with a shorter quasiparticle's 
mean free path. In this case, the high field phase should have a one-dimensional midulation parallel to the field. The 
direction of modulation can be clarified, e.g., through ultrasound measurements of the type performed in Ref.[16(. 

Another main result in this work is the field-induced change of the orientation of nodal planes in the FFLO state 
constructed in n — 1 LL. According to the structural transitions between different vortex lattices illustrated in Fig. 3, 
the nodal planes become perpendicular to the SC layers in some field ranges. In real systems, the nodal planes 
together with the so-called pancake vortices are trapped by point defects becoming active only on the SC layers as 
pinning sites. If the nodal plane are parallel to the SC layers, the pinning effect on the nodal planes is negligible even 
if they sit on the SC alyers on averages. Since a large applied current parallel to the z-direction, i.e., perpendicular 
to the layers, can induce a vortex flow parallel to the layers, the above-mentioned pinning effect should be visible in 
field dependences of out-of-plane resistivity R± (T) data. It is believed that the field dependent oscillatory behavior 
observed in R±(T) data in the field-induced superconductor A-(BETS)2FeCi4 |9| is an evidence of this pinning effect 
of nodal planes induced by structural transitions between different Josephson vortex lattices. This explanation of 
the phenomena is different from an explanation used in Ref.Q based on the scenario in Ref.[l8| where structural 
transitions between the Josephson vortex lattices are assumed to be absent, and the nodal planes are not parallel 
to the SC layers. At the static level, this corresponds to the w — 1 vortex lattice, represented by Fig. 4(b), in the 
regime H C 2 > H cr . As mentioned in relation to Fig.l, however, the modulation of the FFLO state in this situation is 
usually parallel to the applied field. Further, according to the conventional description of a vortex flow based on the 
time-dependent GL equation [l9l ]. the vortex flow is nothing but an uniform flow of the SC order parameter itself. 
Since both the vortices and the nodal planes are parts of the SC order parameter, the assumption of [l8| a vortex 
flow under nodal planes at rest contradicts the conventional description of SC dynamics [Hj]. In contrast, the present 
picture is applied to the ordinary situation with a strong paramagnetic depairing in which H C 2 < H cr and hence, 
with a flat H C 2 curve at high temperatures. In the ordinary layered materials under parallel fields, the structural 
transitions between Josephson vortex lattices are not clearly reflected in resistive data because changes of pinning 
effects accompanying structural changes of Josephson vortex lattices are small. It seems to us that the significant 
oscillatory behavior of resistivity in Rcf. 9] is consistent with a large change of pinning effect due to an orientational 
change of the extended nodal planes. 

Throughout this paper, we have neglected a possibility of a modulation parallel to H within the n — 1 LL vortex 
state. This vortex state with a two-dimensional modulation is expected to occur at lower temperatures in the region 
domianted by the n = 1 LL. Further details of possible phase diagrams will be discussed elsewhere. 
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